Telepresence Control of a Dual-Arm Dexterous Robot 
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ABSTRACT 

Telepresence is an approach to teleoperation 
that provides egocentric , intuitive interactions 
between an operator and a remote environment. 
This approach takes advantage of the natural 
cognitive and sensory-motor skills of an on-orbit 
crew and effectively transfers them to a slave 
robot. A dual-arm dexterous robot operating 
under telepresence control has been developed 
and is being evaluated. Preliminary 
evaluation revealed several important 
observations that suggest the directions of 
future enhancement. 

INTRODUCTION 

The current approaches to robot teleoperation 
in the Space Shuttle as well as the 
International Space Station Alpha (ISSA) are 
based on ’’joystick" type hand controllers. The 
visual feedback is provided by multiple 
cameras, many of which are mounted on the 
robot arms. This approach to teleoperation is 
similar to the cock-pit design of fighter 
aircraft. For manipulator control, this 
approach can be counter-intuitive, and may 
overload the visual and manual capacities of 
the operator. The problem becomes even more 
amplified when the slave robot is not designed 
to reflect the degree of dexterity the human 
operator possesses. As a result, the operator’s 
skill is not effectively transferred to the slave 
robot. A different approach to robot 
teleoperation is telepresence. In telepresence, 
the master control and feedback devices are 
designed to maximize the use of the operator’s 
innate cognitive and sensory-motor skills [1][2]. 


The following describes the Phase I activities 
of an evolving robotics testbed at the NASA 
Johnson Space Center (JSC). The testbed 
system, called the Dexterous Anthropomorphic 
Robotic Testbed (DART), has telepresence as its 
baseline operating mode. Ultimately, DART 
will be operating under shared control , where 
telepresence control of the robot will be 
augmented by intelligent automation. DART is 
controlled by the Full Immersion Telepresence 
Testbed (FITT), the interface to the human 
operator [2]. 

PHASE I OBJECTIVE 

The DART Phase I objective is to develop, 
demonstrate, and optimize a baseline 
telepresence system. The steps to achieve this 
goal include: (1) developing a dexterous 
telerobotic system with telepresence control; (2) 
developing a flexible, modular, shared control 
architecture; and (3) conducting comparative 
evaluation of telepresence versus other types of 
controls. The following will primarily focus on 
the activities leading to the accomplishment of 
Step (1) and (2). Preliminary evaluations, in 
partial fulfillment of Step (3), will also be 
described. 

SYSTEM OVERVIEW 

Most telepresence applications require at least 
two functional components: master and slave. 
The master component is usually the operator’s 
telepresence interface, and the slave is usually 
an emulator of the master. In our setup, FITT 
acts as the master that controls DART, the 
slave robot. The FITT and DART systems are 
shown conceptually in figure 1. 
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Figure 1. Telepresence control of a dual-arm dexterous robot, (a) The Dexterous Anthropomorphic 
Robotic Testbed, (b) The Full-Immersion Telepresence Testbed (concept drawing). 


Full-Immersion Telepresence Testbed 

FITT provides intuitive control of DART. This 
testbed immerses the operator in the robot’s 
environment and links human and robot motions 
and senses as transparently as possible to 
provide a natural feel to the operator. The 
FITT system includes interfaces for controlling 
DART's head camera unit, arms, hands, and 
base. A Virtual Research™ helmet displays 
DART’s stereo camera images with a 100 degree 
field of view. The depth perception provided 
with stereo imaging is one of the testbed s most 
important immersion features. A Polhemus™ 
tracker located on the top of the helmet 
commands the orientation of DART's head 
camera unit. The same type of sensor is also 
attached to each of the operator's wrists, 
providing full 6 degree-of-freedom (DOF) 
control of the robotic arms. EXOS™ hand 
masters worn by the operator provide gripper 
and joint level finger control in dexterous 
teleoperation. 

Dexterous Anthropomorphic Robotic Testbed 

DART, shown in figure 2, includes several 
robotic devices, controllers, and supporting 
workstations. The robotic arms are PUMA 
562's with an 8.8 pound payload capability. 
Each arm also has a force-torque sensor. On the 
right arm is a Stanford /JPL hand. Each finger 
has a urethane fingertip to provide a high 
static friction surface and can be hyper- 
extended to provide a large manipulation 
envelope. On the left arm is a parallel jaw 
gripper. The head camera unit. that provides 
video feedback to the teleoperator supports 3 
DOF rotations and contains two color CCD 
cameras. The driver level software is executed 


on two Tadpole™ multiprocessor systems. 
Each multiprocessor system has four M88000 
processors and runs a multiprocessor version of 
the UNIX operating system. The vision system 
is implemented on a DataCube™ pipeline 
image processor board. 



Figure 2. The Dexterous Anthropomorphic 
Robotic Testbed (DART). 


The DART system is controlled via a modular, 
distributed control architecture as shown in 
figure 3. Each subsystem spans one or more 
processes. The subsystem processes are 
distributed across three separate computers, 
networked on an Ethernet backbone. The 
subsystems communicate and are synchronized 
by high-level communication software called 
the Tele-Robotics Interconnection Protocol 
(TelRIP) [4]. This architecture provides a 
flexible environment for development, 
maintenance, and future enhancements. FITT 
controls DART by linking to this Ethernet 
backbone and commanding the subsystems 
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through TelRIP. The router process, denoted by 
R, is responsible for transmitting data to the 
appropriate subsystem processes. 



Figure 3. DART's distributed control 
architecture. 

PRELIMINARY EVALUATIONS 

Preliminary evaluations of the DART and FITT 
systems are currently being performed using 
operators of varying skill levels, ranging from 
several years of robotic experience to absolutely 
no engineering experience. This allows the 
intuitiveness of operation to be qualitatively 
evaluated. The tasks range from inspection to 
object handling to dexterous manipulation. 

Inspection tasks are comprised mainly of 
bringing an object towards the head camera and 
viewing it from different angles. These tasks 
provide information about the required display 
resolution, stereo perception, as well as the 
effect of working with egocentric views of the 
workspace. The object handling tasks include 
picking up objects of various sizes and shapes 
( e.g ., balls, pipes, tools) and placing them at a 
different location, and handing objects back and 
forth between the dexterous hand and the 
gripper. Some of the dual-hand dexterous tasks 
performed are tying a knot with a rope, folding 
and unfolding a thermal blanket, and 
manipulating an electronic task panel which 
contains toggle and rocker switches, push 
buttons, sliders, and a dial. These tasks reflect 
some of the basic dexterity and skills required 
for on-orbit extra- and intra-vehicular 
activities (EVA/IVA). 

OBSERVATIONS 

One of the most significant observations from 
the preliminary evaluations is the short time 
it takes a new operator to become proficient 


with the system. For example, operators with 
no previous experience were able to transfer 
objects between the two hands and manipulate 
the controls on the panel within a 30 minute 
session. Operators with considerable 
experience in "cock-pit" type control have also 
found the training time greatly reduced due to 
the intuitiveness of the motion controls and the 
immersiveness of the visual feedback. 

The weight of the exoskeleton hand masters 
causes muscle fatigue when the system is used 
for long duration. This limitation presents 
some difficulties when it is necessary to 
maintain a specific position for a long period of 
time. This observation suggests the need for a 
mechanism that will allow the operator to re- 
adjust his or her arm positions (e.g., indexing). 

While teleoperation of the dexterous hand 
offers much flexibility for grasping, it was 
found inadequate for manipulation. The 
difficulty lies in the inability of the operator 
to preshape the hand and execute manipulation 
primitives, such as turning and pinching, in a 
consistent manner. 

The operator can experience mild motion 
sickness when using the system due to a slight 
delay between the motions of the operator's 
head and the DART camera system. This only 
occurs when the operator makes large, quick 
head movements. Motion sickness usually 
occurs whenever there is a significant mismatch 
between the robot's and the operator's rate of 
motion. Motion sickness can also be caused by 
unintended body and head movements. 
However, since the operator rarely has to make 
large head movements once focused on a task, 
this problem is not a major prohibiting factor. 

Although the current system provides the 
necessary visual cues to perform many tasks, a 
few limitations of the visual feedback have 
been observed. The visual feedback the 
operator receives is coarse (320 X 240 pixels) 
and the distance between the head cameras is a 
little too narrow, so the depth perception of the 
operator is not optimal. These visual 
limitations can have serious impacts on the 
operator's performance. For example, since 
FITT currently does not offer force-reflection, 
the operator assesses the force imparted onto 
the environment by watching for the amount of 
physical compliance. The active compliance of 
the DART's fingers is very useful in this regard. 
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Another problematic area encountered is the 
transformation of human hand motions to 
DART’s hand motions. Several transformation 
methods were explored [3]. These methods 
included joint-to-joint mapping, forward and 
inverse kinematics transformations, and a 
combination of joint and Cartesian control. The 
two major difficulties encountered when 
applying these techniques are the dissimilar 
kinematics of the human's and DART’s hands, 
and the slight changes in the sensor positions 
when the gloves are taken off and put back on. 
Joint-to-joint mapping was chosen as the 
method of control due to the computational 
simplicity and the intuitiveness of the control. 

The telepresence evaluations also revealed 
some interesting operator behaviors. For 
example, an initial exercise is desirable before 
each session to familiarize the operator with 
the system's behavior. The exercise typically 
involves having the operator command the 
robot’s arms, hands and head in various 
different ways to explore the dexterity of the 
robot. Without the exercise, less experienced 
operators often have the tendency to move like 
a robot, not fully utilizing his or her natural 
coordination skill. After a few training 
sessions, the operator generally will learn to 
compensate for any kinematics dissimiliarities 
between the operator and the robot. 

FUTURE WORK 

The results of our initial evaluation have 
pointed out several areas for improvement. The 
exoskeleton gloves will be replaced by light- 
weight Cybergloves™ to reduce fatigue, A 
position indexing mechanism will be 
implemented to allow the operator to 
reposition his or her arms while the robot 
remains still. Grasp and manipulation 
primitives modulated by operator's hand 
movements will be developed for tasks that 
require a high degree of accuracy and control. A 
second generation head camera unit will be 
fabricated to provide a tighter head tracking 
and to correct the narrow interpupilary 
distance. A high-resolution (640 X 480 pixels) 
head-mounted displays will be sought to 
improve operator's visual acuity. 

A force-reflective dexterous arm master, 
developed by EXOS™, will be integrated with 
FITT to evaluate the effect of force-reflection. 
Additional evaluations will be conducted to 
quantify the performance of the DART/ FITT 


system. New test subjects will be recruited to 
study the correlation between training-time 
versus performance, and the performance of 
"cock-pit” type control versus telepresence. 

CONCLUSION 

Telepresence is not a new idea. It is, however, 
an idea that is becoming a reality due to the 
recent advances in head-mounted display, 
dexterous glove controller, motion trackers, 
force-reflective masters, and other human 
compatible interactive devices. The DART and 
FITT combination represents an integration of 
these telepresence technologies for space 
robotics applications. Many lessons were 
learned in our preliminary evaluations. While 
several areas for improvement were identified, 
the benefit of telepresence in space robotics is 
clearly evident by the variety of complex tasks 
DART/FITT can perform under the control of an 
operator with minimal training. 
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